Towards Person Google: Multimodal Person Search and Retrieval

With the increasing amount of available multimedia data, efficient systems for searching and retrieving relevant AV documents are needed. Since keyword based indexing is very time consuming and inefficient due to linguistic and semantic ambiguities, content based multimedia retrieval systems have been proposed, that search and retrieve AV documents based on audio and visual features. While content based image retrieval has been a very active research field only some work has been done in the field of person specific search and retrieval, where the goal is to find a AV document with a specific person present within the audio and/or the visual stream. This article describes an original system for multimodal person search (Person Google) within AV documents and provides some initial performance results.